Internet Security Visualization Case Study: Instrumenting a Network for NetFlow Security Visualization Tools

With the development of the Internet and organizational intranets, it has become an increasingly critical and difficult task to monitor large and complex networks indispensable to security risk management and network performance analysis. Monitoring for security situational awareness with visualization has been shown to be an effective and efficient approach. However, the quality of source data for visualization tools directly determines resulting performance. In the security monitoring visualization tools developed at NCSA, diverse log files are employed, the most important ones being Cisco NetFlows and Argus NetFlows. Due to their uniform record format and distinctive level of abstraction over raw packets, NetFlows are increasingly used by security engineers to infer security events. In spite of the wide usage of NetFlows, there has only been limited work on the data management issues of using NetFlows as a unique data source. Although several popular tools have been developed for processing Cisco NetFlows, only NCSA and the University of Chicago have developed processing tools for Argus NetFlows. In addition, several prominent differences exist between Cisco NetFlows and Argus NetFlows. Lastly, with increasingly higher line rates, sampling appears to be the trend for minimizing router overhead and data overload. Sampling mechanisms employed by Cisco and sFlow are introduced, along with discussion of their possible effect on security analysis. This work is expected to provide practical insight into data management issues inherent with the use of NetFlows source data for security and network performance monitoring.